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FOREWORD 


The  research  reported  in  this  paper  was  conducted  by  the  Human 
Factors  in  Tactical  Operations  Technical  Area  of  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ARI) . The  research 
program  has  as  its  objective  the  application  of  scientific  research  to 
principles  of  human  performance  in  complex  man/machine  systems,  in  this 
case  the  optimization  of  helicopter  Nap-of-the-Earth  (NOE)  navigation 
and  control  effectiveness.  The  research  was  conducted  under  Army 
Project  2Q164715A757,  "Aircrew  Performance,"  FY  1974  Work  Program.  ARI 
Research  Report  1190,  Technical  Paper  277,  and  Research  Memorandum  76-26 
also  present  aspects  of  the  initial  Technical  Area  research.  The  ARI 
Field  Unit  since  established  at  Fort  Rucker,  Ala.,  with  Charles  A. 

Gainer  as  chief,  now  conducts  the  research  on  aircrew  performance.  ARI 
Research  Reports  1197,  1198,  and  1199  describe  the  continuing  program. 

As  part  of  the  Aircrew  Performance  work  unit  effort,  this  study  was 
conducted  to  determine  the  level  of  NOE  navigation  proficiency  and  to 
measure,  by  means  of  an  ARI-developed  human  performance  measurement  tech- 
nique, the  effects  of  training  and  experience  on  proficiency  level.  The 
research  is  part  of  ARI's  total  program  of  research  in  support  of  special 
requirements  of  the  Deputy  Chief  of  Staff  for  Operations  (DCSOPS) . It 
was  conducted  with  the  cooperation  of  the  Army  Aviation  Center  (USAAVNC) 
at  Fort  Rucker,  Ala.,  and  the  U.S.  Army  Training  and  Doctrine  Command 
(TRADOC)  and  Forces  Command  (FORSCOM) . Special  acknowledgement  is  made 
to  MG  W.  J.  Maddox,  Jr.,  Commander  of  the  Army  Aviation  Center,  and 
BG  J.  H.  Merryman,  Director  of  Army  Aviation  (during  the  period  in  which 
the  research  was  conducted) , without  whose  support  this  research  would 
not  have  been  possible.  CPT  H.  Cook,  CW2  P.  Ross,  and  CW2  R.  Beatty 
served  as  technical  advisers,  test  pilots,  and  instructors  during  this 
research,  and  CPT  J.  Isenhower  served  as  project  officer  at  Fort  Rucker. 


AN  ASSESSMENT  OF  THE  NAVIGATION  PERFORMANCE  OF  ARMY  AVIATORS  UNDER 
NAP-OF-THE-EARTH  CONDITIONS 


BRIEF 


Requirement : 

The  basic  objective  for  this  series  of  studies  was  to  obtain  empir- 
ical data  on  how  the  NOE  navigational  skill  level  of  Army  aviators  is 
affected  by  (1)  pilot  experience,  defined  as  the  number  of  flight  hours, 
and  (2)  two  levels  of  training — a 7-hour  terrain  analysis  course  and  a 
6-hour  NOE  course.  Other  objectives  were  to  define  a baseline  on  pilot 
navigation  proficiency  and  to  develop  a field  research  methodology  to 
measure  pilot  performance  in  NOE  flight. 


Procedure : 

A total  of  35  Army  rotary  wing  aviators  participated  in  three  field 
experiments  in  which  they  flew  representative  NOE  navigation  missions  in 
UH-1H  aircraft.  Their  flight  experience  was  200-2,700  hours  for  14 
experienced  pilots,  1,000-3,750  hours  for  14  instructor  pilots,  and  200 
hours  for  7 Aviation  School  graduates.  Half  received  the  special  train- 
ing. The  aviators  were  assigned  missions  in  which  designated  landing 
zones  had  to  be  found  for  simulated  medical  evacuations  or  supply  deliv- 
eries. All  35  aviators  navigated  at  least  six  NOE  routes  ranging  from 
23  to  25  kilometers  (km)  in  length.  Twenty-eight  of  the  aviators  were 
also  tested  on  aircraft  control  and  the  performance  of  various  NOE 
maneuvers . 


Findings : 

NOE  navigation  is  a trainable  skill.  Experienced  aviators  with 
training  performed  better  than  experienced  aviators  without  training. 

In  addition,  recent  graduates  with  15  hours  of  NOE  training  performed 
better  than  experienced  aviators  without  training. 

Specialized  training  in  NOE  navigation  is  valuable.  The  instructor 
pilots  indicated  that  experience  is  more  important  in  aircraft  control 
than  in  navigation.  The  group  with  an  average  of  1,380  flight  hours 
controlled  their  aircraft  better  than  the  group  with  200  hours,  but  this 
difference  in  experience  appeared  to  have  no  effect  on  the  navigation 
task . 


The  field  research  measures  and  techniques  developed  for  these 
experiments  seemed  well  suited  for  NOE  navigation  research.  The  objec- 
tive mission  success  score  (OMSS) , a composite  performance  measure  indi- 
cating the  probability  that  the  mission  would  be  successfully  completed, 
proved  useful  and  has  a .75  correlation  with  subjective  rating  by  expert 
NOE  instructor  pilots.  As  expected,  it  was  shown  that  NOE  navigation  is 
an  extremely  difficult  and  complex  task.  The  OMSS's  show  that  NOE 
missions  flown  over  the  Fort  Rucker  terrain  resulted  in  an  overall  prob- 
ability success  of  .63. 


Implications  of  Findings: 

All  pilots  need  more  NOE  navigation  training,  which  should  include 
a more  systematic  academic  NOE  navigation  training  course,  both  in 
initial  entry  training  and  at  operational  units.  Academic  training 
should  emphasize  checkpoint  selection  and  identification,  since  most  of 
the  variance  in  the  OMSS's  is  attributable  to  errors  in  locating  landing 
zones.  This  recommended  training  could  utilize  multimedia  methods  (e.g., 
film,  tape  recordings,  slides)  and  should  emphasize  practical  exercises 
rather  than  standard  lecture  techniques. 

Because  of  its  sensitivity  to  navigation  performance,  the  OMSS 
should  be  further  refined  and  validated  for  use  throughout  Army  aviation 
in  evaluating  and  diagnosing  NOE  navigation  proficiency. 
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AN  ASSESSMENT  OF  THE  NAVIGATION  PERFORMANCE  OF  ARMY  AVIATORS  UNDER 

NAP-OF-THE-EARTH  CONDITIONS 


INTRODUCTION 

In  response  to  the  highly  sophisticated  air  defense  threat  of  a 
potential  enemy,  the  U.S.  Army  has  refined  a terrain  flight  tactic  for 
use  by  its  aviation  units.  This  tactic  has  been  designated  Nap-of-the- 
Earth  (NOE)  flight  (i.e.,  flight  at  treetop  level  and  below),  using  ex- 
isting terrain  features  to  mask  the  aircraft  from  enemy  radar  and 
optical  detection.  This  mode  of  flying  will  enhance  survivability  and 
mission  effectiveness  near  the  forward  edge  of  the  battle  area. 

The  increased  use  of  NOE  tactics  presents  a number  of  problems  not 
previously  encountered  by  Army  rotary  wing  aviators.  The  primary  prob- 
lem is  in  navigation.  The  difficulty  in  navigation  can  be  attributed  to 
several  factors: 

1.  The  limited  forward  view  when  operating  in  close  proximity  to 
the  ground. 

2.  The  perspective  at  which  checkpoints  are  viewed. 

3.  The  design  of  aerial  charts,  which  does  not  take  into  account 
navigation  from  the  NOE  altitudes. 

To  improve  aviator  NOE  navigation  ability,  a research  effort  was 
initiated  first,  and  then  familiarization  programs  were  introduced  into 
the  Initial  Entry  Rotary  Wing  (IERW)  program  and  were  implemented  at  op- 
erational units.  Once  changes  to  the  training  program  were  established, 
they  had  to  be  evaluated  and  revised,  as  necessary.  Further,  for  evalu- 
ations to  be  meaningful  and  their  results  comparable,  a standardized 
assessment  technique  was  required.  Because  the  traditional  performance 
measurement  techniques  for  aviators  did  not  readily  adapt  to  the  NOE 
situation^,  owing  to  the  lack  of  standardization,  development  of  such  a 
standardized  technique  was  one  objective  of  the  research. 

This  report  describes  three  separate  experiments,  all  performed  to 
measure  the  following: 

1.  The  effectiveness  of  present  NOE  training  as  shown  by  the  pi- 
lot's ability  to  navigate  his  aircraft  over  unfamiliar  terrain 
at  NOE  altitude, 


Farrell,  John  P.  Measurement  Criteria  in  the  Assessment  of  Helicopter 
Flight  Performance,  presented  at  conference  on  Aircrew  Performance  in 
Army  Aviation,  Fort  Rucker,  Alabama,  November  1973. 


1 


2.  The  effect  of  additional  terrain  analysis  training  on  NOE 
navigation, 

3.  The  effect  of  flight  experience  on  NOE  navigation  ability,  and 

4.  The  utility  and  applicability  of  the  measurement  techniques 
used  in  the  field  environment. 

These  experiments  were  preliminary  and  were  designed  to  be  an 
empirical  evaluation  of  the  navigation  problem.  The  work  reported  here 
was  conducted  from  24  September  1973  to  20  December  1974  at  the  U.S. 

Army  Aviation  Center  (USAAVNC) , then  the  Aviation  School;  it  was  the 
first  step  in  direct  support  to  the  Deputy  Chief  of  Staff  for  Operations 
(DCSOPS)  operational  objective:  "to  determine  the  most  effective  type 
and  amount  of  training  needed  to  develop  satisfactory  proficiency  in 
NOE-type  flying." 


METHOD 


Subjects 


The  subjects  in  this  study  were  35  Army  helicopter  pilots  who  were 
currently  proficient  in  the  UH-1H  helicopter  and  had  some  exposure  to 
NOE  flight,  at  either  entry  or  unit  level.  The  pilots  were  selected  to 
represent  three  different  experience  levels.  The  14  pilots  in  Experi- 
ment 1 were  selected  to  represent  the  general  population  of  Army  UH-lH 
helicopter  pilots;  their  experience  ranged  from  200  to  2,700  flight 
hours . 

The  14  subjects  in  Experiment  2 were  selected  to  obtain  a sample  of 
the  more  proficient,  instructor-level  UH-lH  helicopter  pilots;  their 
flight  experience  ranged  from  1,000  to  3,750  flight  hours. 

The  seven  subjects  in  Experiment  3 were  selected  as  representative 
of  recent  graduates  of  Army  Aviation  School  who  had  also  recently  com- 
pleted a new  15-hour  course  in  NOE  navigation.  These  pilots  had  accrued 
200  flight  hours. 

Prior  NOE  experience  was  more  difficult  to  specify.  The  200-hour 
pilots  in  Experiment  3 did  have  a 15-hour  course  in  NOE  navigation,  but 
the  NOE  experience  of  pilots  in  Experiments  1 and  2 varied.  All  were 
required  to  have  had  at  least  some  unit  training  in  NOE.  However,  the 
extent  and  the  definition  of  the  training  varied  widely. 


Facilities 

Test  Range.  The  test  range  used  in  all  experiments  is  located  west 
of  Troy,  Ala.,  and  is  shown  on  standard  (1:50,000  scale)  map  sheets 
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designated  Lapine  (3747  IV)  and  Youngblood  (3747  I).  The  area  is  heavily 
wooded,  with  numerous  small  fields  and  open  areas  and  many  small  streams 
and  low-lying  marsh  areas  throughout.  The  land  itself  is  gently  rolling 
terrain  that,  by  consensus,  presents  a very  difficult  NOE  navigation 
problem. 

The  range  was  divided  into  three  areas  of  operation;  each  area  was 
further  divided  into  four  routes,  each  containing  an  initial  point  (IP), 
intermediate  landing  zones  (LZ's),  and  a release  point  (RP)  designating 
the  end  of  the  mission. 

Aircraft.  Two  UH-1H  aircraft  were  employed  in  the  study.  The 
first,  designated  the  "low  ship,"  was  used  to  transport  the  subject 
pilot,  instructor/safety  pilot,  and  the  ARI  test  supervisor.  The  second, 
designated  the  "high  ship,"  served  two  purposes:  it  flew  "chase"  for 
safety  reasons,  and  it  provided  the  high-altitude  platform,  at  approxi- 
mately 800  feet,  from  which  a second  instructor  pilot  could  track  the 
flight  of  the  low  ship  for  experimental  purposes. 

Visual  Aids.  The  following  materials  were  used  in  the  terrain 
analysis  training  provided  during  the  study:  (1)  Petrey  1:50,000 
Experimental  Map  Sheet,  Day  Use,  Series  V744  AMD,  Sheets  3747  I and  IV, 
and  (2)  35imn  color  slides  of  the  three  areas  of  operation. 

Test  Personnel.  Four  Army  aviators  were  required  to  conduct  the 
tests.  Two  were  highly  qualified  NOE  instructor  pilots.  One  acted  as 
pilot  in  the  low  ship  and  the  subject  pilot  served  as  navigator.  The 
other  instructor  pilot  served  as  tracking  observer  in  the  high  ship. 

His  role  was  to  follow  the  flight  of  the  low  ship  and  trace  its  ground 
track  on  a map  in  relation  to  the  prescribed  course  previously  drawn  on 
the  map.  The  other  two  aviators  alternated  in  flying  the  high  ship. 

The  two  instructor  pilots  also  served  as  technical  advisers  during  design 
and  implementation  of  these  tests. 

Safety.  All  provisions  of  USAAVNC  Supplement  I to  Army  Regulation 
95-1,  385-10  and  40  were  strictly  observed. 


Experimental  Design 

Experiment  1.  The  independent  variables  were  the  presence  or  ab- 
sence of  terrain  analysis  training  and  the  level  of  flight  experience. 

To  determine  the  effects  of  these  variables  on  inflight  navigation 
skill,  seven  pairs  of  subjects  were  systematically  matched  for  experi- 
ence. One  member  of  each  pair  was  randomly  assigned  to  an  experimental 
group  and  the  other  to  a control  group.  The  control  group  (A)  flew  12 
flights  in  three  phases.  Each  phase  was  flown  over  a different  area  so 
that,  although  a pilot  may  have  become  somewhat  familiar  with  the  general 
area  of  operations  (AO),  he  never  flew  the  same  route  twice.  The 
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experimental  group  flew  eight  flights,  standing  down  between  Phase  I and 
Phase  III  for  a 2-day  terrain  analysis  course  prepared  and  presented  by 
the  instructor  pilots.  The  course  consisted  of  a 3-hour  review  of  a pro- 
gramed text  on  navigation  and  a 3-hour  terrain  association  exercise. 

During  this  part  of  the  course  the  subject  was  shown  slides  of  different 
points  in  each  AO  and  asked  to  report  his  position  on  a standard  1:50,000 
scale  map. 

To  control  for  differences  in  the  terrain  composition  in  each  AO, 
the  order  in  which  the  areas  were  overflown  was  systematically  varied 
between  subjects.  For  example.  Subject  1 flew  areas  I,  II,  III  in  that 
order,  whereas  Subject  3 flew  areas  III,  II,  I.  The  effects  of  the  inde- 
pendent variables  were  measured  in  terms  of  navigation  proficiency. 

Experiment  2.  The  purpose  of  Experiment  2 was  to  validate  the  tech- 
niques and  methodology  used  in  Experiment  1 and  to  expand  the  subject 
population.  The  14  subjects  in  Experiment  2 were  all  highly  qualified 
NOE  instructor  pilots  or  had  served  as  members  of  a Modern  Army  Selected 
Systems  Test,  Evaluation  and  Review  (MASSTER)  evaluation  team  of  Chief 
of  Development  Evaluation  Command  (CDEC)  "night  owl"  team.  The  back- 
ground and  design  of  this  experiment  were  essentially  the  same  as 
Experiment  1,  with  the  following  exceptions: 

1.  Pilots  were  assumed  to  be  at  a higher  skill  level, 

2.  The  variable  of  flight  experience  was  deleted, 

3.  The  number  of  flights  was  reduced  to  10  (i.e.,  3 over  each  area 
of  operations  and  1 aircraft  handling  flight) , and 

4.  Both  control  and  experimental  groups  navigated  over  the  same 
number  of  flights. 

Experiment  3.  The  purpose  of  Experiment  3 was  to  measure  the  effec- 
tiveness of  the  15-hcur  NOE  navigation  course  that  had  replaced  the  orig- 
inal 6-hour  NOE  familiarization  sequence.  The  subjects  used  in  this  study 
were  the  most  recent  (Dec.  1974)  graduates  of  the  USAAVNS  Initial  Entry 
Rotary  Wing  (IERW)  program.  Each  of  these  pilots  had  taken  the  15-hour 
NOE  sequence.  These  pilots  flew  six  flights,  i.e.,  two  over  each  of  three 
areas  of  operation.  The  design  was  otherwise  identical  to  Experiment  2. 


Procedures 


Upon  being  selected,  each  subject  was  given  an  information  pamphlet 
containing  a description  of  the  purpose  and  design  of  the  study,  along 
with  administrative  information.  Upon  reporting  to  the  project  officer 
at  Fort  Rucker,  the  subject  was  thoroughly  briefed  again  and  was  asked 
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to  fill  out  a biographical  data  sheet.  Prior  to  each  mission,  the  sub- 
ject received  a premission  briefing  at  the  Lowe  Heliport  Field  Office. 
The  briefing  was  conducted  by  the  Senior  Instructor  Pilot  and  consisted 
of  three  parts: 

1.  A presentation  and  discussion  of  the  day's  mission, 

2.  Map  analysis  of  the  route  to  be  followed,  and 

3.  A review  of  flight  safety  procedures,  if  necessary. 

The  subject  pilot  received  his  NOE  route  assignment  to  be  flown 
that  day.  The  appropriate  map  sheet  was  provided  with  the  route  marked 
in  yellow,  and  with  the  IP,  LZ,  and  RP  designated.  An  identical  map 
was  given  to  the  observer  who  flew  in  the  high  ship. 

On  a typical  flight,  the  subject  pilot,  instructor  pilot,  and  ARI 
test  supervisor  manned  the  low  ship.  The  subject  pilot  then  navigated 
at  an  altitude  of  800  ft.  from  the  point  of  hover  check  to  the  IP  of  the 
NOE  route,  with  the  high  ship  flying  chase.  The  subject  pilot's  identi- 
fication of  the  IP  was  scored  by  the  instructor  pilot  as  correct  or 
incorrect . 

The  subject  pilot  then  began  flying  the  route  at  NOE  altitude, 
following  as  accurately  as  possible  the  prescribed  course  on  the  map. 

The  actual  course  flown  by  the  subject  pilot  was  recorded  by  the  high 
ship  observer.  The  subject  was  required  to  identify  and  stop  at  each 
intermediate  LZ.  The  subject  pilot's  selection  of  each  LZ  was  scored 
by  the  instructor  pilot  as  correct  if  the  landing  was  within  100  meters 
of  the  correct  landing  zone. 

At  the  end  of  the  mission  the  instructor  pilot  took  control  of  the 
aircraft  and  flew  to  the  refueling  point  at  Troy  Army  Airfield.  Here 
the  subject  was  debriefed,  shown  the  actual  track  flown,  and  told  where 
he  went  off  course.  The  subject  pilot  was  asked  to  review  the  mission 
and  to  indicate  the  reasons  for  navigational  decisions.  After  the 
debriefing,  the  high  ship  observer's  map  was  retained  by  the  ARI  test 
supervisor.  Most  of  the  dependent  measures  were  derived  from  this  map. 

After  a short  rest  of  approximately  an  hour,  the  experimental 
group  flew  the  second  mission  of  the  day. 

Samples  of  inflight  intercommunications  during  Experiment  1 were 
recorded  by  the  ARI  observer  for  evaluation  purposes  and  further 
research  into  inflight  communication  techniques.  Observations  of  the 
subjects’  navigational  procedures  were  also  recorded  (see  the  Conclu- 
sions section  of  this  report).  After  each  flight,  the  instructor  pilot 
subjectively  evaluated  the  effectiveness  of  the  mission  in  terms  of 
three  categories:  complete  success,  partial  success,  or  failure. 
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Data 


At  the  end  of  each  flight  the  following  data  were  collected  by  the 
ARI  test  supervisor: 

1.  The  annotated  map  containing  the  subject's  actual  track  and 
other  measures  of  checkpoint  identification  (see  Figure  1 for 
a representative  map) , 

2.  The  instructor  pilot's  evaluation  of  mission  effectiveness, 
and 

3.  Behavioral  observations  and  critical  times  during  the  mission. 


Dependent  Measures 


The  following  set  of  dependent  measures  was  derived  from  these  data 
and  used  in  all  three  experiments  to  measure  navigation  performance. 

Mission  Success  Scores.  Two  measures  of  mission  success  were  em- 
ployed: a subjective  mission  success  score  (SMSS)  and  an  objective 

mission  success  score  (OMSS) . 

1.  SMSS — As  discussed  under  "Procedures,"  the  instructor  pilot  was 
asked  to  rate  the  subject's  navigational  effectiveness  in  terms 
of  three  categories:  complete  failure,  indicated  by  zero;  par- 
tial success,  indicated  by  1;  and  complete  success,  indicated 
by  2.  This  rating  considered  any  evidence  of  disorientation 
along  the  route  and  any  assistance  that  had  to  be  given  to  the 
pilot  by  the  instructor.  For  example,  if  the  instructor  had 

to  fly  the  subject  back  to  the  correct  route  because  the  pilot 
became  completely  disoriented,  a failure  rating  of  zero  would 
be  given. 

This  rating  was  then  divided  by  2 (the  criterion  for  complete 
success),  yielding  a measure  ranging  from  .00  to  1.00.  If  the 
subject  was  completely  successful  on  all  flights,  he  would 
receive  a rating  of  1.00.  A mean  subjective  mission  success 
score  across  all  flights  for  each  subject  was  then  calculated. 

2.  OMSS — This  measure  was  constructed  to  serve  as  a composite 
metric  representative  of  the  subject's  scores  on  four  individ- 
ual measures  (see  Figure  2).  The  four  measures  used  were 
number  of  IP's  missed,  number  of  LZ's  missed,  number  of  250- 
meter  excursions  from  the  course  line,  and  number  of  1,000- 
meter  excursions.  Each  of  the  four  measures  was  weighted  on 
the  basis  of  responses  gathered  from  a questionnaire  distributed 
to  15  veteran  combat  pilots. 
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W1  W2  W3  W4 

SD  X1  + SD  X2  + SD  X3  + SD  X4 
12  3 4 


OMSS  = 1 


where  X equals  the  number  of  occurrences  in  each  error  category: 
Xx  = IP  failures, 

X2  = LZ  failures, 

X3  = 250-meter  excursions,  and 
X^  = 1,000-meter  excursions; 

and  W equals  the  empirically  determined  weight  of  each  error: 

Wx  = 9, 

W2  = 9, 

W3  = 3,  and 


therefore : 


= K =22.50, 

SD1  1 ' 


S5J-  K2  = 25*71' 


— = K3  = 4.76,  and 


SD"  = K4  = 18-91; 

4 

and  F equals  a score  which  represents  complete  mission  failure: 


Note:  If  the  actual  sum  of  the  weighted  error  scores  exceeded  100,  the 

OMSS  was  still  scored  as  zero. 


Figure  2.  Calculation  of  OMSS. 
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The  responses  were  then  tabulated  and  a mean  weight  for  each 
error  measure  was  calculated.  The  weights  were  as  follows: 

Failure  to  find  IP — 9, 

Failure  to  find  LZ — 9, 

250-meter  excursion — 3,  and 
1,000-meter  excursion — 7. 

Each  weight  (W)  was  then  adjusted  by  dividing  by  the  standard 
deviation  (SD)  of  the  raw  error  scores  under  each  category 
across  all  subjects.  Next,  the  number  of  errors  (X)  in  each 

category  was  determined  for  each  flight.  The  number  of  errors 

in  each  category  was  then  multiplied  by  the  associated  ad- 
justed weight.  The  weighted  totals  were  summed  across  cate- 
gories and  this  sum  was  then  divided  by  100  (F) ; the  value  of 
100  was  chosen  as  a convenience  and  because  scores  of  100  would 
represent  total  mission  failure,  since  the  OMSS  is  based  on  the 
number  of  errors. 

The  formula  is  as  follows: 

W1  W2  W3  W4 

SD  X1  + SD  X2  + SD  X3  + SD  X4 

12  3 4 

OMSS  = 1 - 


F 

The  result  of  this  calculation  was  subtracted  from  1 to  indi- 
cate the  objective  mission  success  score  (see  Figure  2) . 

SMSS  and  OMSS  are  composite  measures:  they  tend  to  reflect  the 
totality  of  the  subject's  navigational  performance.  The  measures  are 
"part"  measures:  they  describe  individual  elements  of  the  total  mission. 

Probability  of  Finding  IP  (PIP) . This  measure  was  determined  by 
dividing  the  number  of  IP's  found  by  the  total  number  of  IP's  that  ex- 
isted across  all  flights.  For  example,  if  Subject  1 flew  9 flights 
and  found  5 IP's,  the  PIP  would  be  5/9  = .55. 

Probability  of  Finding  LZ  (PLZ) . This  measure  was  calculated  by 
dividing  the  number  of  correctly  found  LZ's  by  the  total  number  of 
possible  LZ's  across  all  flights  for  that  subject.  For  example,  if 
Subject  1 correctly  found  15  LZ's  out  of  a total  of  23,  the  PLZ  would 
be  15/23  = .65. 

250-Meter  Excursions  (250  EXC) . This  measure  indicates  the  number 
of  times  that  the  pilot  deviated  from  the  center  of  the  specified  course 
line  by  more  than  250  meters  but  less  than  1,000  meters.  The  number  of 
such  excursions  was  summed  over  all  flights  for  each  subject. 
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1,000-Meter  Excursions  (1,000  EXC) . This  measure  indicates  the 
number  of  times  the  pilot  deviated  from  the  center  of  the  prescribed 
course  line  by  more  than  1,000  meters.  The  number  of  such  excursions 
was  summed  across  all  flights  for  each  subject. 

Total  Distance  Traveled  (TDT) . This  is  the  distance  covered  by 
the  pilot  over  all  flights  measured,  including  all  excursions,  from  the 
IP  to  the  PIP. 

Percentage  of  Distance  Traveled  Off-Course  250  Meters  (PDTOC  250) . 
This  measure  relates  the  number  of  250  EXC  to  the  TDT.  It  is  calculated 
by  dividing  the  distance  traveled  during  all  250  EXC  summed  over  all 
flights  by  the  TDT  for  each  subject.  For  example,  if  Subject  A flew  a 
TDT  of  168,000  meters  and  traveled  16,800  meters  in  250  EXC,  his  PDTOC 
would  be  10%. 

Percentage  of  Distance  Traveled  Off-Course  1,000  Meters  (PDTOC 
1 , 000) . This  measure  relates  the  number  of  1,000  EXC  to  the  TDT  and  is 
calculated  the  same  as  PDTOC  250. 

Percentage  of  Distance  Traveled  On-Course  (PDTOnC) . This  measure 
was  calculated  in  order  to  provide  a one-number  indicator  of  the  con- 
tinuous measure  of  navigation.  For  example,  if  Subject  A had  a PDTOC 
250  of  10%  and  PDTOC  1,000  of  6%;  the  PDTOnC  =1.00  -10%  -6%  or  84%. 

Time.  This  is  the  mean  time  in  minutes  across  all  flights  for  each 
subject  calculated  from  the  IP  to  the  RP  or  ended  at  the  discretion  of 
the  test  supervisor. 


RESULTS 

The  results  of  these  studies  are  presented  as  individual  experi- 
ments. The  statistical  test  for  significance  used  in  most  cases  was  the 
Mann  Whitney  U2 , because  many  of  the  measures  were  ordinal  rather  than 
interval.  The  Mann  Whitney  U retains  most  of  the  power  of  the  t-test, 
while  not  constraining  the  researcher  to  the  latter's  assumptions.  The 
Mann  Whitney  U is  most  valuable  when  samples  are  small  and  the  under- 
lying distribution  is  not  necessarily  normal. 


Experiment  1 

Overall  Performance  Level.  The  data  in  Table  1 represent  mean 
scores  for  14  subjects  across  all  flights  for  each  of  the  navigation 


Siegel,  S.  Nonparametric  Statistics  for  the  Behavioral  Sciences. 
New  York:  McGraw-Hill  Book  Co.,  1956. 
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measures  described  in  the  Procedure  Section.  The  OMSS  varies  from  .38 
to  .79  with  a mean  of  .57.  The  SMSS  mean  was  .54  with  scores  varying 
from  .17  to  .79.  The  mean  PIP  and  mean  PLZ  for  all  subjects  was  .76  and 
.69,  respectively. 

Two  measures  were  used  to  quantify  the  distance  traveled  off- 
course.  These  measures  approximate  the  500-  and  2,000-meter  corridors; 
breaking  these  corridors  results  in  250-  and  1,000-meter  excursions 
respectively.  Table  1 indicates  that  the  Army  aviators  involved  in  the 
study  made  approximately  twelve  250  meter  excursions  and  four  1,000- 
meter  excursions  during  the  course  of  7 flights. 

The  present  distance  flown  outside  the  500-meter  acceptable  corri- 
dor (PDTOC  250)  for  all  pilots  was  12.80%.  The  mean  PDTOC  1,000  of 
5.94%  is  smaller.  Pilots  who  made  mistakes  of  this  magnitude,  and  were 
unable  to  correct  their  errors  within  a 5-minute  period,  were  considered 
by  the  instructor  pilot  as  lost  and  were  vectored  back  to  the  course 
line. 


The  PDTOC  was  81.27%.  This  means  that  a pilot  returning  from  the 
forward  edge  of  battle  area  (FEBA)  and  required  to  maintain  a 500- 
meter  corridor  would  have  an  18%  chance  of  straying  outside  the  500- 
meter  corridor. 

Terrain  Analysis  Training  Effect.  The  navigation  data  presented 
in  Table  2 reflect  the  performance  of  those  pilots  in  the  control  group, 
(no  terrain  analysis  training)  and  those  in  the  experimental  group  (with 
terrain  analysis  training) . To  control  for  the  additional  flying  experi- 
ence gained  by  the  control  group  (which  did  not  stand  down  for  training) , 
Phase  II  of  the  control  group  flights  was  compared  with  Phase  III  of  the 
experimental  group.  The  means  for  each  subject  are  therefore  based  on 
three  flights  rather  than  on  all  flights  measured. 

The  data  in  Table  2 show  that  the  group  which  received  map  training 
found  the  checkpoints  (PIP  and  PLZ)  more  often  and  stayed  within  the 
1,000-meter  corridor  more  often  than  did  the  group  without  map  training. 
This  better  performance  is  reflected  in  the  two  overall  measures:  the 
trained  group  had  an  SMSS  of  .60  and  an  OMSS  of  .59,  whereas  the  un- 
trained group  scored  .50  on  SMSS  and  .51  on  the  OMSS.  Although  the  dif- 
ferences are  not  statistically  significant  at  the  conventional  .05  level 
of  confidence,  there  is  an  apparent  trend,  which  indicates  support  for 
the  hypothesized  benefits  of  additional  map  training. 

There  is  a statistically  significant  difference  (p  = .006)  in  the 
ability  of  the  two  groups  to  find  checkpoints.  The  trained  group  had 
a .86  PIP,  and  the  untrained  group  scored  .67.  The  significance  level 
is  .006. 
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Table  1 

Data  for  All  Subjects  for  All  Flights  in  Experiment 


Table  2 


Comparison  of  Groups  With  and  Without  Terrain  Analysis  Training 

in  Experiment  1 


2 A NAV  Measure 


Subjects 

SMSS 

OMSS 

pip 

PLZ 

250 

EXC 

1,000 

EXC 

PDTOC 

250 

PDTOC 

1,000 

PDTOnC 

TIME 

Pilots 

with 

training 

N = 7 

.60 

.59 

.86 

.75 

5.74 

.71 

16.95 

3.04 

79.99 

28.67 

Pilots 

without 

training 

N = 7 

.50 

.51 

.67 

.63 

3.  71 

2.28 

10.11 

9.57 

80.30 

32.23 

Mann 

Whitney 

U test 

ns 

ns 

* ★ 

ns 

ns 

* 

*p  < .05. 
**p  < .01. 


The  terrain-analysis-trained  sample  took  4 minutes  less,  on  the 
average,  to  complete  their  missions  and  flew  8,000  meters  less  than  the 
nontrained  sample.  This  indicates  that  they  stayed  closer  to  the  course 
line  than  the  other  group.  This  is  further  supported  by  the  fact  that 
the  nontrained  group  flew  6%  more  of  its  total  distance  off-course  by 
more  than  1,000  meters  than  did  the  trained  group  (see  Column  PDTOC 
1,000).  However,  none  of  these  differences  was  statistically  significant. 

Effects  of  Experience.  Table  3 presents  the  navigation  data  from 
Table  1,  grouped  according  to  experience  level.  The  high-experience 
group  had  a mean  flight  hour  experience  level  of  1,387  hours,  and  the 
low-experience  group  had  214  hours. 

On  first  inspection  of  these  data,  the  results  appear  paradoxical. 

The  low-experience  group  did  better  in  all  measures  than  the  high- 
experience  group.  However,  this  observation  must  be  qualified  by  the 
very  small  differences  between  the  groups. 
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Table  3 


Comparison  of  the  High-Experience  Group  (1,387  mean  flight  hours)  With 


the  Low 

-Experience 

Group 

(214  mean 

flight 

hours) 

in  Experiment 

1 

3 A 

NAV  Measure 

Subjects 

SMSS 

OMSS 

PIP 

250 

PLZ  EXC 

1,000 

EXC 

PDTOC 

250 

PDTOC 

1,000 

PDTOnC 

TIME 

High- 

experi- 

ence 

pilots 

N = 7 

.51 

.56 

.74 

.68  13.70 

3.71 

15.69 

6.47 

77.85 

29.67 

Low- 

experi- 

ence 

pilots 

N = 7 

.57 

.59 

.79 

.70  9.71 

3.57 

9.91 

5.39 

84.69 

28.76 

Mann 

Whitney 

U test 

ns 

ns 

ns 

ns  * 

ns 

*P  < .05. 


Experiment  2 

In  Experiment  2,  14  highly  experienced  NOE  instructor  pilots  were 
evaluated  in  terms  of  their  navigation  ability.  The  design  and  proce- 
dures of  this  experiment  were  the  same  as  those  for  Experiment  1,  with 
the  exceptions  noted  earlier  in  the  Method  section. 

Comparison  Measures.  Table  4 compares  the  navigation  scores  of  the 
NOE  instructor  pilots  to  those  of  the  "average"  aviators  in  Experiment  1. 
The  scores  of  the  NOE  instructor  pilots  in  five  of  the  six  measures  are 
significantly  better  than  those  of  average  pilots  (Mann  Whitney  U, 
p < .05).  The  instructor  pilots  seem  to  have  maintained  course  more 
efficiently,  as  indicated  by  the  higher  percentage  of  distance  traveled 
on  course.  They  did,  however,  make  significantly  more  250-meter  excur- 
sions than  the  average  group.  This  did  not  adversely  affect  the  per- 
centage of  distance  traveled  on  course,  because  the  extent  of  these 
excursions  was  not  as  great  as  the  extent  of  excursions  committed  by 
the  pilots  in  Experiment  1. 
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Table  4 


Comparison  of  Scores  Between  "Average"  Pilots  and  NOE  Instructor  Pilots 

in  Experiment  2 


4 A NAV  Measure 

250 

1,000 

PDTOC 

PDTOC 

Subjects 

SMSS 

OMSS 

PIP 

PLZ 

EXC 

EXC 

250 

1,000 

PDTOnC 

TIME 

NOE 

Instruc- 

tor 

pilots 

N = 14 

.75 

.65 

.93 

.79 

18.00 

3.40 

10.80 

3.76 

85.44 

30.00 

Average 

pilots 

N = 14 

.54 

.57 

.76 

.09 

11.71 

3.64 

12.80 

5.94 

81.27 

29.22 

Mann 

Whitney 

U test 

* 

** 

** 

★ 

★ 

ns 

*p  < .05. 

**p  < .01. 


Experiment  3 

This  experiment  was  designed  to  compare  the  navigation  performance 
of  two  groups  of  recent  IERW  graduates  uncontaminated  by  varying  levels 
and  types  of  operational  experience.  The  first  group  (Sept.  1973  gradu- 
ates) received  only  the  6-hour  NOE  familiarization  sequence,  whereas  the 
other  (Dec.  1974  graduates)  received  a 15-hour  NOE  navigation  course. 

Comparison  Measures.  Although  there  is  not  complete  concurrence 
among  the  navigation  measures  used  in  Table  5,  there  is  a general  ten- 
dency in  favor  of  the  15-hour  group.  The  15-hour  group  is  superior  to 
the  6-hour  group  on  8 of  the  10  measures  used,  and  on  2 of  these  mea- 
sures the  effect  is  statistically  significant.  The  OMSS  for  the  15-hour 
group  is  .70  and  the  OMSS  for  the  6-hour  group  is  .60.  The  15-hour 
group  has  a .90  probability  of  finding  the  IP,  and  the  6-hour  group 
achieves  only  a .78  probability.  Both  of  these  differences  are  statis- 
tically significant  (Mann  Whitney  U,  p < .05). 
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Table  5 


Comparison  of  Performance  Scores  Between  200-Hour  IERW  Graduates 
With  15  Hours  NOE  Training  and  Those  With  6 Hours  NOE  Familiarization 

Experiment  3 


5 A 

NAV  Measure 

Subjects 

SMSS 

OMSS 

PIP 

PLZ 

250 

EXC 

1,000 

EXC 

PDTOC 

250 

PDTOC 

1,000 

PDTOnC 

TIME 

Pilots 

with 

15-hr 

training 

N = 7 

.70 

. 70 

.90 

.87 

15 

2.33 

10.52 

2.69 

86.79 

24.86 

Pilots 

with 

6-hr 

training 

N = 6 

.54 

.60 

.78 

.74 

7 

3.16 

9.69 

6.23 

84.08 

28.61 

Mann 

Whitney 

U test 

ns 

* 

* 

ns 

** 

ns 

*p  < .05. 

**p  < .01. 


A somewhat  anomalous  finding  is  that  on  2 of  the  10  measures  (fre- 
quency of  250  EXC  and  PDTOC  250)  the  6-hour  group  was  superior  to  the 
15-hour  group.  The  250  EXC  was  statistically  significant  (p  < .01)  but 
the  difference  on  the  PDTOC  250  was  quite  small,  with  the  15-hour  group 
having  only  about  1%  more  distance  off-course  than  the  6-hour  group. 

This  effect  is  difficult  to  account  for;  a probable  explanation  is  that 
the  15-hour  group  made  more  minor  excursions  because  they  were  traveling 
at  a higher  airspeed.  The  fact  that  the  15-hour  group  covered  the  dis- 
tance in  less  time,  on  the  average,  indicates  that  errors  were  discov- 
ered more  quickly  and  corrected  more  effectively  than  were  the  errors 
made  by  the  6-hour  group. 

Combined  Results.  The  scores  for  all  35  pilots  were  combined.  The 
sample  did  represent  a wide  range  of  flight  experience  (200-3,600  hours) 
and  NOE  training  (6  hours  NOE  familiarization  to  qualification  as  NOE 
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instructor  pilot) . In  generalizing  from  these  results,  the  user  must  be 
cautious,  as  several  possible  factors  could  have  influenced  the  data  of 
this  small  sample.  For  example,  the  14  instructors  could  have  inflat- 
able data.  In  addition,  the  restricted  terrain  sets  limits  on 
generalization. 

The  statistic  employed  for  the  purpose  of  generalization  is  the 
standard  error  of  the  mean  and  associated  95%  confidence  limits.  This 
allows  inference  from  the  sample  to  the  total  population  within  speci- 
fied limits  of  certainty.  Table  6 presents  the  sample  means,  standard 
errors,  and  95%  confidence  limits  for  all  navigation  measures. 

The  data  in  Table  6 indicate  that  the  aviators  achieved  on  OMSS  of 
.63,  with  a standard  error  of  .0259.  The  confidence  interval  ranged 
from  .58  to  .68.  This  indicates  that  5%  of  the  time  the  mean  OMSS  for  a 
similar  population  of  Army  aviators  falls  somewhere  in  this  interval. 

The  SMSS  can  be  interpreted  similarly.  The  sample  mean  SMSS  is  .65  with 
a standard  error  of  .0361  and  a confidence  interval  extending  from  .58 
to  .72.  Each  of  the  other  navigation  scores  may  be  interpreted  in  a 
similar  manner. 


Table  6 

Mean  Performance  Scores,  Standard  Errors,  and  Associated  Confidence 
Intervals  for  All  Pilots  (N  = 35)  Over  All  Flights  (242) 


6 A NAV  Measure 

250  1,000  PDTOC  PDTOC 

Subjects  SMSS  OMSS  PIP  PLZ  EXC  EXC  250  1,000  PDTOnC  TIME 


Mean 

N = 35 

.65 

.63 

.85 

.77 

14.86 

3.23 

11.44 

4.32 

84.32 

29.00 

Standard 

error 

.0361 

.0259 

.0281 

.0024 

1.0217 

.4122 

.8468 

.6681 

1.2938 

.8579 

Confi- 

dence 

limit 

.58- 

.58- 

.80- 

.73- 

12.86- 

2.42- 

9.78- 

2.97- 

81.78- 

27.32- 

(95%) 

.72 

.68 

.90 

.81 

16.86 

4.04 

13.10 

6.67 

86.86 

30.68 

Table  7 subdivides  the  35  subjects  into  five  groups  of  7 subjects 
each.  The  breakdown  was  conducted  along  two  variables — training  and 
experience.  This  categorization  is  primarily  for  summary  purposes,  as 
the  groups  were  previously  compared  in  terms  of  the  individual  experi- 
ments. When  interpreting  these  data,  one  must  remember  that  the 
results  from  the  first  experiment  were  based  on  three  flights  per  sub- 
ject, whereas  the  results  for  the  other  experiment  were  based  on  at 
least  six  flights  per  subject.  The  five  groups  as  distributed  by  exper- 
iment are  as  follows: 

Experiments 

1.  Average  aviators  with  on-the-job  training, 

2.  Average  aviators  without  on-the-job  training, 

3.  NOE  instructor  pilots  with  on-the-job  training, 

4.  NOE  instructor  pilots  without  on-the-job- 
training,  and 

5.  Average  aviators  with  NOE  navigation  training  during  IEKW  I 
program. 

In  Table  7,  these  groups  are  compared  in  terms  of  navigation 
scores.  For  the  OMSS,  the  data  show  that  the  average  aviators  with 
early  NOE  training  tend  to  perform  more  effectively  than  the  other 
groups.  They  also  show  the  highest  probability  of  finding  the  LZ  (.85), 
in  comparison  with  other  types  of  aviators  tested.  The  fact  that  this 
component  is  weighted  heaviest  in  regard  to  mission  success  accounts 
for  the  high  OMSS  for  average  aviators  with  early  NOE  training,  as  com- 
pared to  the  NOE  instructor  pilots  with  training  who  scored  .64  on  OMSS, 
but  only  .76  on  finding  the  LZ. 

For  the  SMSS,  the  NOE  instructor  pilots  with  training  scored  high- 
est (.82)  of  the  five  groups,  and  the  instructor  pilots  without  train- 
ing were  next  with  .80. 

In  finding  the  IP  of  each  mission,  the  instructor  pilots  without 
additional  training  scored  .95. 
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Subjects  SMSS  OMSS  PIP  PLZ 


Avg 

pilots 

with 

training  .60  .59  .86  .75 

Avg 

pilots 
with  no 

training  .50  .51  .67  .63 

Instruc- 

tor 

pilots 

with 

training  .82  .64  .91  .76 

Instruc- 

tor 

pilots 

no 

training  .80  .63  .95  .81 

Avg 

pilots 
15-hr 
NOE 


70 


.70 


90  .87 


Table  7 


Performance  Scores 
iups  of  NOE  Aviators 


7 A NAV  Measure 


250 

EXC 

1,000 

EXC 

PDTOC 

250 

PDTOC 

1,000 

PDTOnC 

TIME 

5.14 

.71 

16.95 

3.04 

79.99 

28.67 

3.71 

2.28 

10.11 

9.57 

80.30 

32.23 

15.00 

2.00 

10.63 

2.47 

86.90 

29.57 

21.00 

4.85 

10.41 

4.97 

84.62 

30.00 

15.00 

2.33 

10.52 

2.09 

86.79 

28.61 

DISCUSSION 


Effectiveness  of  Existing  NOE  Training 

The  first  and  most  obvious  implication  from  the  results  of  these 
experiments  is  that  Army  pilots  require  specialized  NOE  navigation 
training.  Statements  pertaining  to  the  mission  effectiveness  of  this 
performance  in  a tactical  situation  must  be  qualified,  since  the  test 
situation  cannot  represent  combat.  If  the  OMSS  is  a valid  score,  it  is 
assumed  that  the  Army  would  prefer  that  the  average  OMSS  for  navigation 
would  approach,  even  if  it  does  not  reach,  1.00  rather  than  the  .63 
which  represented  actual  accomplishment  (see  Tables  1,  4,  6,  and  7). 

Checkpoint  identification  (map/terrain  association)  appears  to  be 
the  primary  and  most  critical  error  made  in  NOE  navigation.  This  find- 
ing is  based  upon  the  fact  that  15  operational  combat  pilots  who  rated 
the  importance  of  the  various  NOE  elements  weighted  the  two  errors 
dealing  with  checkpoint  identification  as  the  most  critical  to  success- 
ful mission  completion.  On  a scale  of  10,  the  mean  weights  for  failure 
to  find  the  IP  and  LZ  were  both  9. 

It  appears,  from  the  overall  navigation  scores  and  from  the  mea- 
ures  of  checkpoint  identification,  that  the  major  cause  of  low  naviga- 
tion scores  is  the  lack  of  ability  to  identify  and  associate  terrain 
features  with  points  on  a map. 

It  was  originally  thought  (based  on  Experiment  1)  that  correct 
identification  of  checkpoints  and  greater  congruency  in  map/terrain 
association  would  result  in  improved  course  maintenance  performance. 

An  aviator  who  has  the  objective  LZ  firmly  located  by  means  of  map/ 
terrain  association  can  devote  more  time  and  effort  to  maintaining  the 
prescribed  route  and  staying  within  a specified  corridor. 

This  hypothesis  was  not  supported  by  the  results  of  later  experi- 
ments. It  appears  (see  Table  7)  that  as  aviators  get  better  at  check- 
point identification,  they  tend  to  tolerate  deviation  from  the  nominal 
course  line.  Improved  checkpoint  identification  ability  promotes  con- 
fidence. Added  confidence  tends  to  promote  a selection  of  the  best 
route  more  readily. 


Experience  Effects 

Experience  in  flight  at  higher  altitudes  does  not  appear  to  have 
transfer  to  navigation  at  NOE  altitude.  In  Experiment  1,  the  experi- 
enced pilots  without  specialized  NOE  training  did  not  perform  any  better 
than  the  graduates  with  only  200  flight  hours  (see  Table  3) . 
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It  is  also  evident  from  Table  3 that  flight  experience,  per  se, 
does  not  necessarily  improve  identification  of  checkpoints.  This  sug- 
gests that  the  identification  skill  could  be  taught  in  a classroom  or 
part-task  simulation  environment. 

It  is  hypothesized  that  the  pilots  with  low  flight  time  performed 
at  least  as  well  as  those  with  high  flight  time  because  they  had 
recently  received,  as  part  of  their  undergraduate  tactics  course,  an 
NOE  flight  familiarization  sequence.  The  high-time  pilots  had  received 
their  NOE  training  on  the  job  at  the  unit  level. 

When  all  the  subgroups  are  compared  (see  Table  7) , the  recent 
graduates  of  the  15-hour  NOE  course  actually  show  higher  OMSS  than  do 
the  instructor  pilots  with  or  without  specialized  NOE  training.  It  is 
likely  that  the  extensive  experience  in  navigating  at  high  altitudes 
did  not  generalize  to  the  NOE  situation  because  of  the  very  different 
visual  environment  at  NOE  altitude. 

The  data  in  Table  7 indicate  that  specialized  training  is  warranted 
for  proficiency  at  NOE  navigation.  It  is  evident  from  these  findings 
that  experience  alone  is  not  the  answer  to  the  problem  of  NOE  navigation 
performance  enhancement. 


Terrain  Analysis  Training  Effects 

The  administration  of  the  experimental  map-study  course  in 
Experiment  1 resulted  in  slightly  improved  navigation  performance. 

Comparison  of  map-trained  subjects  with  non-map-trained  subjects 
suggests  that  checkpoint  identification/selection  skill  does  improve 
with  this  training.  It  is  possible  that  specific  training  applied  to 
terrain  analysis  could  increase  overall  navigation  performance  (see 
Table  1,  PLZ) . 

Improvement  in  the  ability  to  associate  map  and  terrain  should 
yield  improvement  in  overall  proficiency  scores.  This  skill  is  train- 
able,  as  evidenced  by  the  data  in  Table  2.  Checkpoint  identification 
ability  does  improve  as  a function  of  map  training.  Table  2 shows 
that  experimental  training  of  average  aviators  improved  their  ability 
to  find  the  IP  by  19%  and  their  ability  to  find  the  LZ  by  12%.  Since 
this  gain  in  performance  was  a function  of  an  admittedly  crude  and 
general  map-study  course,  a highly  concentrated  course  structured 
specifically  to  teach  checkpoint  identification  might  raise  this  skill 
level  even  more. 
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Training  Suggestions 


Because  the  thrust  of  the  NOE  flight  research  performed  by  ARI  is 
to  assist  in  the  development  of  effective  NOE  training  methods,  this 
report  would  not  be  complete  without  suggesting  ways  of  improving  NOE 
navigation . 

It  has  been  demonstrated  that  the  basic  problem  facing  pilots  in 
NOE  flight  is  navigation — a conclusion  which  will  come  as  no  surprise 
to  operational  pilots.  The  experiments  described  in  this  report  indi- 
cate that  the  basic  problem  in  NOE  navigation  is  terrain  analysis,  a 
skill  which  has  two  aspects — mission  planning  and  inflight  terrain 
analysis.  Both  of  these  aspects  should  be  given  additional  emphasis  in 
IERW  and  operational  training. 

Inflight  training  of  terrain  analysis  skills  is  difficult  because 
of  the  limited  opportunity  to  stop  and  review  environmental  inputs  and 
to  practice  analytic  skills.  Thus,  it  appears  that  much  of  the  needed 
terrain  analysis  training  should  be  given  in  ground  academic  training. 

Based  upon  both  the  research  findings  as  well  as  the  personal 
observation  of  the  ARI  investigators,  it  is  suggested  that  ground  aca- 
demic terrain  analysis  training  should  consist  of  the  following 
elements : 

1.  Training  in  the  basic  principles  of  topography  and  cartography, 

2.  Performance-oriented  exercises  in  interpretation  of  maps,  with 
special  emphasis  on  analysis  of  terrain  features  and  their  representa- 
tion on  standard  maps  used  in  flight, 

3.  Practice  in  NOE  mission  planning  with  emphasis  on  identifica- 
tion of  checkpoints  and  use  of  terrain  for  masking,  and 

4.  Performance-oriented  exercises  in  NOE  navigation,  making  use 
of  motion  picture  films  of  routes  flown  at  NOE. 

Such  a training  program,  which  should  make  use  of  all  available 
media,  (film,  35mm  slides,  tape  recordings,  vugraphs,  paper-and-pencil 
exercises,  etc.)  should  consume  at  least  10  working  days.  Land  naviga- 
tional problem  exercises  would  be  performed  either  concurrently  or 
immediately  upon  conclusion  of  the  terrain  analysis  course.  This  would 
then  be  followed  by  5 days  of  inflight  practice  in  actual  navigation 
under  controlled  flight  conditions. 

The  adequacy  of  such  a training  approach  toward  improving  NOE  navi- 
gational skills  should  be  evaluated  by  using  techniques  such  as  those 
already  developed  and  described  in  this  report. 
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Additional  Observations 


Intracrew  Conw  uni cat ions ■ Based  on  behavioral  observations  in 
flight  and  preliminary  review  of  some  of  the  intercom  tapes,  it  is  felt 
that  pilot/navigator  communication  is  often  inadequate  for  NOE  naviga- 
tion. Two  areas  of  deficiency  are  noted — infrequency  of  communications 
and  misinterpretation  of  information.  A number  of  subjects  had  to  be 
repeatedly  prompted  by  the  pilot  to  provide  vectors  for  flight.  Co- 
pilots acting  as  navigators  often  fail  to  give  the  pilot  advance  infor- 
mation about  the  checkpoints  they  are  looking  for,  so  that  the  pilot  can 
act  as  a second  pair  of  eyes.  Terminology  in  communicating  information 
between  pilot  and  copilot  is  not  standardized.  In  general,  there  is  a 
lack  of  formal  communication  procedure  between  copilot  and  pilot.  The 
development  of  a standardized  procedure  should  be  explored  as  a means 
of  improving  performance  at  NOE  navigation. 

Subjective  Responses  of  Pilots.  Based  on  subjective  reports 
collected  during  mission  debriefings,  it  is  felt  that  the  workload  on 
the  NOE  navigator  is  very  high  and  accordingly  stressful. 

Individual  differences  in  capability,  possibly  of  a perpetual  na- 
ture, may  also  play  a role.  This  hypothesis  needs  further  research. 

Standardized  Measurement  System.  The  lack  of  a standardized  per- 
formance measurement  technique  for  NOE  has  been  noted.  The  utility  of 
the  OMSS  in  this  study  indicates  that  this  metric  could  be  further 
standardized  and  evaluated  as  a possible  basic  measurement  and  diag- 
nostic technique  in  maintaining  NOE  navigator  proficiency. 

Characteristics  of  Expert  NOE  Navigators.  Observations  by  test 
personnel  flying  with  the  subjects  suggest  that  the  successful  NOE  navi- 
gator performs  with,  among  others,  the  following  capabilities. 

1.  Uses  a series  of  multiple  checkpoints  or  terrain  features  as 
a basis  for  determining  whether  or  not  he  is  on  the  correct  route. 

Many  navigators  look  for  only  a single  checkpoint,  which  is  more  diffi- 
cult to  find  than  a combination  of  checkpoints.  When  they  fail  to  find 
the  single  checkpoint,  they  are  more  likely  to  persevere  on  an  incor- 
rect track.  Successful  navigators  tend  to  recognize  more  quickly  that 
they  are  deviating  from  the  specified  route  and  reverse  their  direction 
of  flight  quickly.  As  a result,  their  deviations  from  the  specific 
route  are  shorter. 

2.  Can  judge  more  accurately  che  distance  traveled  in  terms  of 
time.  Navigators  tend  to  estimate  the  distance  they  must  travel  to 
reach  a particular  checkpoint,  but  they  do  this  by  subjective  time. 

When  their  time-distance  relationship  is  inaccurate,  they  tend  to 
become  confused  with  regard  to  the  recognition  of  checkpoints,  because 
they  either  anticipate  these  in  advance  of  their  appearance  or  they 
overrun  them. 
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3.  Warns  the  pilot  in  advance  of  the  checkpoints  he  is  looking 
for,  so  that  the  pilot  can  be  looking  for  them  also. 
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APPENDIX 


USARI  WEIGHTING  INSTRUMENT 


The  following  are  representative  errors  encountered  in  the  Army 
Research  Institute  performance  measures  of  NOE  navigation  ability. 

Your  help  as  operational  combat  pilots  is  necessary  to  attach  the  pro- 
per importance  to  these  types  of  errors  so  that  we  may  concentrate  our 
research  effort  in  the  most  effective  direction.  Therefore,  please 
assign  a value,  from  1-10,  to  each  of  these  errors,  reflecting  their 
impact  on  combat  mission  success.  The  higher  the  number  assigned,  the 
more  serious  the  error.  For  example,  if  you  were  to  assign  a value  of 
10  to  a 250-meter  deviation,  this  would  indicate  that  a mistake  of  this 
type  would  very  seriously  compromise  combat  mission  success.  If  you 
were  to  assign  a 1,  it  would  indicate  that  an  error  of  this  type  would 
have  little,  if  any,  impact  on  the  success  of  a combat  mission. 

The  error  categories  are  as  follows: 

Failure  to  find  or  identify  an  LZ  other  than  the  IP. 

A 250-meter  deviation  from  a nominal  course  line. 

Failure  to  find  the  IP. 

A 1,000-meter  deviation  from  a nominal  course  line. 
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1 US  WAC  Ctr  & Sch.  Ft  McClellan.  ATTN:  Lib  1 

1 US  WAC  Ctr  & Sch.  Ft  McClellan.  ATTN:  Tng  Dir  1 

1 USA  Quartermaster  Sch.  Ft  Lee.  ATTN:  ATSM  TE  1 

f Intelligence  Material  Dev  Ofc.  BWL,  Ft  Holabird  1 

1 USA  SE  Signal  Sch.  Ft  Gordon.  ATTN:  ATSO-EA  1 

1 USA  Chaplain  Ctr  & Sch.  Ft  Hamilton.  ATTN:  ATSC-TE-RD  1 

1 USATSCH.  Ft  Sustis,  ATTN:  E due  Advisor  1 

1 USA  War  College,  Carlisle  Barracks.  ATTN:  Lib  1 

2 WRAIR,  Neuropsychiatry  Div  2 

1 DLI,  SDA,  Monterey  1 

1 USA  Concept  Anal  Agcy.  Bethesda,  ATTN:  MOCA-WGC  1 

1 USA  Concept  Anal  Agcy.  Bethesda.  ATTN:  MOCA-MR  1 

1 USA  Concept  Anal  Agcy,  Bethesda.  ATTN:  MOCA-JF  1 

1 USA  Artie  Test  Ctr.  APO  Seattle.  ATTN:  STEAC  MO-ASL  1 

1 USA  Artie  Test  Ctr.  APO  Seattle.  ATTN:  AMSTE-PL  TS  1 

1 USA  Armament  Cmd,  Redstone  Arsenal.  ATTN:  ATSKTEM  1 

1 USA  Armament  Cmd.  Rock  Island.  ATTN:  AMSAR-TDC  1 

1 FAA  NAFEC.  Atlantic  City.  ATTN:  Library  1 

1 FAA  NAFEC.  Atlantic  City.  ATTN:  Hum  Eny  Br  1 

1 FAA  Aeronautical  Ctr,  Oklahoma  City.  ATTN:  AAC-44D  2 

2 USA  Fid  Arty  Sch.  Ft  Sill.  ATTN:  Library  2 

1 USA  Armor  Sch.  Ft  Knox,  ATTN:  Library  1 

1 USA  Armor  Sch.  Ft  Knox.  ATTN:  ATSB-Of-E  1 

1 USA  Armor  Sch,  Ft  Knox,  ATTN:  ATSB-DT-TP  1 

1 USA  Armor  Sch,  Ft  Knox,  ATTN:  ATSB-CD-AD  1 


HQUSACDEC,  Ft  Ord,  ATTN:  Library 

HQUSACDEC,  Ft  Ord.  ATTN  ATEC  EX  E-Hum  Factors 

USAEEC,  Ft  Benjamin  Harrison,  ATTN.  Library 

USAPACDC,  Ft  Benjamin  Harrison.  ATTN:  ATCP-HR 

USA  Comm  Elect  Sch,  Ft  Monmouth,  ATTN:  ATSN-EA 

USAEC.  Ft  Monmouth,  ATTN:  AMSEL-CT- HOP 

USAEC,  Ft  Monmouth,  A iTN:  AMSEL  -PA -P 

USAEC.  Ft  Monmouth.  ATTN:  AMSEL-SI-  CB 

USAEC,  Ft  Monmouth,  ATTN:  C,  Facl  Dev  Br 

USA  Materials  Sys  Anal  Agcy,  Aberdeen,  ATTN:  AMXSY-P 

Edgewood  Arsenal,  Aberdeen,  ATTN:  SAREA-BL-H 

USA  Ord  Ctr  & Sch.  Aberdeen.  ATTN:  ATSL-TEM-C 

USA  Hum  Engr  Lab.  Aberdeen,  ATTN:  Library/Dir 

USA  Combat  Arms  Tng  Bd,  Ft  Benning,  ATTN:  Ad  Supervisor 

USA  Infantry  Hum  Rsch  Unit,  Ft  Benning,  ATTN:  Chief 

USA  Infantry  Bd,  Ft  Benning,  ATTN:  STEBC-TE-T 

USASMA,  Ft  Bliss,  ATTN:  ATSS-LRC 

USA  Air  Def  Sch,  Ft  Bliss.  ATTN:  ATSA  -CTD-ME 

USA  Air  Def  Sch,  Ft  Bliss,  ATTN:  Tech  Lib 

USA  Air  Oef  Bd.  Ft  Bliss,  ATTN.  FILES 

USA  Air  Def  Bd,  Ft  Bliss.  ATTN:  STEBD-PO 

USA  Cmd  & General  Stf  College,  Ft  Leavenworth,  ATTN:  Lib 

USA  Cmd  & General  Stf  College,  Ft  Leavenworth.  ATTN:  ATSW  ~SE  - L 

USA  Cmd  & General  Stf  College,  Ft  Leavenworth,  ATTN:  Ed  Advisor 

USA  Combined  Arms  Cmbt  Dev  Act,  Ft  Leavenworth.  ATTN:  DepCdr 

USA  Combined  Arms  Cmbt  Dev  Act,  Ft  Leavenworth,  ATTN:  CCS 

USA  Combined  Arms  Cmbt  Dev  Act,  Ft  Leavenworth,  ATTN:  ATCASA 

USA  Combined  Arms  Cmbt  Dev  Act.  Ft  Leavenworth,  ATTN:  ATCACO-E 

USA  Combined  Arms  Cmbt  Dev  Act,  Ft  Leavenworth,  ATTN:  ATCACC-Cl 

USAECOM,  Night  Vision  Lab,  Ft  Belvoir,  ATTN:  AMSEL— NV— SD 

USA  Computer  Sys  Cmd,  Ft  Belvoir,  ATTN:  Tech  Library 

USAMERDC,  Ft  Belvoir.  ATTN:  STSFB-DQ 

USA  Eng  Sch.  Ft  Belvoir,  ATTN:  Library 

USA  Topographic  Lab,  Ft  Belvoir,  ATTN:  ETL  -TD-S 

USA  Topographic  Lab,  Ft  Belvoir,  ATTN:  STINFO  Center 

USA  Topographic  Lab,  Ft  Belvoir.  ATTN:  ETL-GSL 

USA  Intelligence  Ctr  & Sch,  Ft  Huachuca.  ATTN:  CTD-MS 

USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATS-CTD-MS 

USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATSI-TE 

USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATSI-TE X-GS 

USA  Intelligence  Ctr  & Sch,  Ft  Huachuca.  ATTN:  ATSI-CTS-OR 

USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATSI-CTD— DT 

USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATSI-CTD-CS 

USA  Intelligence  Ctr  & Sch,  Ft  Huachuca.  ATTN:  DAS/SRD 

USA  Intelligence  Ctr  & Sch,  Ft  Huachuca.  ATTN:  ATSI-TEM 

USA  Intelligence  Ctr  & Sch,  Ft  Huachuca.  ATTN:  Library 

CDR,  HQ  Ft  Huachuca.  ATTN:  Tech  Ref  Div 

CDR,  USA  Electronic  Prvg  Grd,  ATTN:  STEEP-MT-S 

CDR,  Project  MASSTER,  ATTN:  Tech  Info  Center 

Hq  MASSTER.  USATRADOC.  LNO 

Research  Institute,  HQ  MASSTER,  Ft  Hood 

USA  Recruiting  Cmd,  Ft  Sherdian,  ATTN:  USARCPM-P 

Senior  Army  Adv.,  USAFAGOD/TAC,  Elgin  AF  Aux  Fid  No.  9 

HO  USARPAC,  DCSPER,  APO  SF  96558,  ATTN:  GPPE-SE 

Stimson  Lib.  Academy  of  Health  Sciences,  Ft  Sam  Houston 

Marine  Corps  Inst.,  ATTN.  Dean -MCI 

HQUSMC,  Commandant.  ATTN.  Code  MTMT  51 

HQUSMC,  Commandant.  ATTN:  Code  MPI-20 

USCG  Academy,  New  London,  ATTN:  Admission 

USCG  Academy,  New  London,  ATTN:  Library 

USCG  Training  Ctr.  NY,  ATTN:  CO 

USCG  Training  Ctr.  NY.  ATTN.  Educ  Svc  Ofc 

USCG.  Psychol  Res  Br.  DC.  ATTN:  GP  1/62 

HQ  Mid-Range  Br,  MC  Det,  Quantico,  ATTN:  P&S  Div 


1  US  Marine  Corps  Liaision  Ofc,  AMC,  Alexandria,  ATTN:  AMCGS-F 
1 USATRADOC,  Ft  Monroe,  ATTN:  ATRO  ED 
6 USATRADOC,  Ft  Monroe,  ATTN:  ATPR  AD 
1 USATRADOC,  Ft  Monroe.  ATTN:  ATTS-EA 

1 USA  Forces  Cmd,  Ft  McPherson.  ATTN:  Library 

2 USA  Aviation  Test  Bd,  Ft  Rucker,  ATTN:  STEBG-PO 

1  USA  Agcy  for  Aviation  Safety,  Ft  Rucker,  ATTN:  Library 
1 USA  Agcy  for  Aviation  Safety.  Ft  Rucker,  ATTN:  Educ  Advisor 
1 USA  Aviation  Sch,  Ft  Rucker,  ATTN:  PO  Drawer  O 

1 HQUSA  Aviation  Sys  Cmd.  St  Louis.  ATTN:  AMSAV-ZDR 

2 USA  Aviation  Sys  Twt  Act..  Edwards  AFB.  ATTN:  SAVTE-T 
1 USA  Air  Def  Sch.  Ft  Bliss.  ATTN:  ATS  A TEM 

1 USA  A.r  Mobility  Rsch  & Dev  Lab,  Moffett  Fid,  ATTN:  SAVDL-AS 
1 USA  Aviation  Sch,  Res  Tng  Mgt,  Ft  Rucker.  ATTN:  ATST-T-RTM 
1 USA  Aviation  Sch.  CO.  Ft  Rucker.  ATTN:  ATST-D-A 
1 HQ.  OARCOM.  Alexandria.  ATTN:  AMXCD-TL 
1 HQ,  DARCOM,  Alexandria.  ATTN  CDR 
1 US  Military  Academy,  West  Point,  ATTN:  Serials  Unit 
1 US  Military  Academy.  West  Point,  ATTN:  Ofc  of  Milt  Ldrshp 
1 US  Military  Academy.  West  Point,  ATTN  MAOR 
1 USA  Standardization  Gp.  UK.  FPO  NY.  ATTN:  MASE-GC 
1 Ofc  of  Naval  Rsch.  Arlington,  ATTN:  Code  452 

3 Ofc  of  Naval  Rsch.  Arlington,  ATTN:  Code  458 
1 Ofc  of  Naval  Rsch,  Arlington.  ATTN:  Code  450 
1 Ofc  of  Naval  Rsch,  Arlington,  ATTN  Code  441 

1 Naval  Aerospc  Med  Res  Lab,  Pensacola,  ATTN:  Acous  Sch  Div 
1 Naval  Aerospc  Med  Res  Lab.  Pensacola,  ATTN:  Code  L51 
1 Naval  Aerospc  Med  Res  Lab,  Pensacola,  ATTN:  Code  L5 
1 Chief  of  NavPers.  ATTN  Pers  OR 
1 NAVAIRSTA.  Norfolk.  ATTN:  Safety  Ctr 
1 Nav  Oceanographic.  IX,  ATTN  Code  6251,  Charts  8r  Tech 
1 Center  of  Naval  Anal.  ATTN  Doc  Ctr 
1 Nav ArrSysCom,  ATTN  AIR  531 3C 
1 Nav  BuMed.  ATTN:  713 
1 NavHelicopterSubSqua  2.  FPO  SF  96601 
1 AFHRL  (FT)  William  AFB 
1 AFHRL  (TT)  Lowry  AFB 

1 AFHRL  (AS)  WPAFB.  OH 

2 AFHRL  (DOJZ)  Brooks  AFB 

1 AFHRL  (DOJN)  Lackland  AFB 
1 HOUSAF  (INYSO) 

1 HOUSAF  (DPXXAI 

1 AFVTG  (RD)  Randolph  AFB 

3 AMRL  (HE)  WPAFB,  OH 

2 AF  Inst  of  Tech.  WPAFB.  OH,  ATTN:  ENE/SL 
1 ATC  (XPTD)  Randolph  AFB 

1 USAF  AeroMed  Lib,  Brooks  AFB  (SUL  -4).  ATTN:  DOC  SEC 
1 AFOSR  (NL).  Arlington 

1 AF  Log  Cmd.  McClellan  AFB.  ATTN:  ALC/DPCRB 

1 Air  Force  Academy.  CO,  ATTN:  Dept  of  Bel  Sen 
5 NavPers  & Dev  Ctr.  San  Diego 

2 Navy  Med  Neuropsychiatric  Rsch  Unit.  San  Diego 
1 Nav  Electronic  Lab,  San  Diego,  ATTN:  Res  Lab 

1 Nav  TrnqCen,  San  Diego,  ATTN:  Code  9000- Lib 
1 NavPostGraSch.  Monterey,  ATTN:  Code  55Aa 
1 NavPostGraSch.  Monterey.  ATTN:  Code  2124 
1 NavTrngEquipCtr,  Orlando,  ATTN:  Tech  Lib 
1 US  Dept  of  Labor,  DC.  ATTN:  Manpower  Admin 
1 US  Dept  of  Justice,  DC,  ATTN:  Drug  Enforce  Admin 
1 Nat  Bur  of  Standards.  DC.  ATTN:  Computer  Info  Section 
1 Nat  Clearing  House  for  MH- Info.  Rockville 
1 Denver  Federal  Ctr.  Lakewood,  ATTN:  BLM 
1 2 Defense  Documentation  Center 

4 Dir  Psych.  Army  Hq,  Rusaell  Ofcs,  Canberra 

1 Scientific  Advsr,  Mil  Bd,  Army  Hq,  Russell  Ofcs,  Canberra 
1 Mil  and  Air  Attache.  Austrian  Embassy 

1 Centre  de  Recherche  Des  Far. tours,  Humame  de  la  Defense 
Netionafe,  Brussels 

2 Canadian  Joint  Staff  Washington 

1 C/Air  Staff.  Royal  Canadian  AF.  ATTN:  Pers  Std  Anal  Br 

3 Chief.  Canadian  Def  Rsch  Staff.  ATTN:  C/CRDS(W) 

4 British  Def  Staff,  British  Embassy,  Washington 


1 Def  & Civil  Inst  of  Enviro  Medicine.  Canada 
1 AIR  CRESS,  Kensington.  ATTN:  Info  Sys  Br 
1 Militaerpsykologisk  T janes  to,  Copehagen 
1 Military  Attache.  French  Embassy,  ATTN:  Doc  Sec 
1 Medecin  Chef.  C.E.R.P.A. -Arsenal,  Toulon/Naval  France 
1 Prin  Scientific  Off.  Appl  Hum  Engr  Rsch  Div.  Ministry 
of  Defense,  New  Delhi 

1 Pers  Rsch  Ofc  Library,  AKA,  Israel  Defense  Forces 
1 Minister  is  van  Defensie,  DOOP/KL  Afd  Sociaal 
Psychologische  Zaken,  The  Hague,  Netherlands 
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